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and  seTnSr  °bjeCtS  Wlth  fmte  SUpport  the  twin  concepts  of  primary 
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1.  Introduction 


=HSSi s^i-SS3S» 

^rrscf"t°r as 

domain  t ISZSTSSS?  ^Tf  ^ 

SSw^ttSsrSsSws 

a*  %***** m  -  «.  Jsstsz  ns?:55^£ 

the  if.L,r  ^0”d  Phenomena  is  the  extrapolation  of  the  measured  Fourier  data  outside  of 

^^StSL^KST" (1L  ^  s“"™  -  ~ S 

There  are  two  schools  of  thought  on  superresolution  by  data  inversion  hereafter  referred 
to  stmply  as  superresolution.  People  who  subscribe  to  the  first  schooled LSb!  ieve£ 

(dfTd  35  3CCUratC  papulation  of  the  measmed  tta  mom  £ 
an  incremental  amount  beyond  the  measurement  bandwidth  for  measured  data  with  realistic 

baSSri7efao^sSi^^)  aDd  SPaTbandWidth  Pr°dUCtS  (SBPs)  £l°) is  unachievable  and 
inieresf  b.ri  *  signal-to-noise  analyses  earned  out  under  the  assumption  that  the  obiect  of 

Schon  75  ,  SUPP“  “  the,image  domain  HI.  t6].  The  definition  of  SBP  will  be  given  in 

SoC  that  the  oWtv  /”  vJC<*  “  regi°D  “  **  imaSe  do"«m  outside  of  wllch  it  is 
k  r  ?,bj  t  m.tensities  are  zero.  People  who  subscribe  to  the  second  school  of 

SNR?  Ld  SBPs  7  loTd^  S(Uperi?S<?Iution  is  P°ssib!e  for  measured  data  with  realistic 
t  SBI“  *  °  jind  baSe  the,r  bellef  on  tbe  appearance  of  structure  in  the  Fourier 

correlated  S^eFn^  “J??  °f  *e  measuremcnt  bandwidth  that  appears  to  be 

correlated  with  the  Founer  structure  of  the  true  object  [3]  [7] 

nr;m!nrithis  Paper’  We  describe  a  Fourier-domain  decomposition  of  superresolved  data  into 

4  wl  stow  ifjfn  "Tfaf\unbiased"  wi“  •*  “■>  qmmSfiTm  sS„ 

jhe  flrSt  SchoDl  of  thought  is  based  upon  the  ability  to  achieve  primary 

thou^tS?sUtb7L7tiDo77bc°?tin|t>0b  seconbary  superresolution,  while  tire  second  school  of 
upon  the  ability  to  achieve  secondary  superresolution.  The  analvsis 
tooughout  this  paper  is  carried  out  in  the  context  of  achieving  superresolution  inThe  estSate 

re, L  -DftShUpport'COnfained  °SCt’  Where  *»  measured  **  *  contained  alowmass 
region  of  the  Fourier  domam.  This  problem  formulation  is  particularly  convenient  Wause 
the  inverse  problem  can  be  expressed  analytically.  In  Section  2  we looW 
mverse  problems  in  terms  of  their  eigenvectoiigenvalue 


inverse  problem  based  upon  the  properties  of  the  eigenvectors  and  eigenvalues  In  Section  I 

variances  finite.  The  bias  properties  of  the  superresolved  data  are  then  explored  as  a  function 
of  fc  amount  of  extrapolation  and  the  numbe?  of  terms  included  inSSSSo?  SS 

nar,™,.C?mClIi.'.e,denrPOS,,IOn  of  suPe™olved  Fourier  data  into  primary  and  secondary 
In  Action  5  DrmmytV"  Sec“on  4-  PtoPMies  of  primary  supericsolution  are  described 
future  woSrare^Sed  m  sTcSS  ”=  ^  F™%.  — - 

2.  Forward  model 

?S^Cti0n’  Tati0nS  f0r  the  forward  311(1  “verse  Pr°blems  are  given.  Particular  attention 
is  placed  upon  the  eigenvector/eigenvalue  expansion  of  the  inverse  problem  in  the  Fourier 
domain.  Properties  of  these  eigenfunctions  and  eigenvalues  are  presented  in  order  to  ram 
rnsight  into  the  properties  of  the  variances  of  unbiased  estimates  of  the  true  object’s  Fourier 

thptepipifieXPri0n!  glVen  f°r  these  variaoces  in  terms  of  an  infinite  summation  involving 
the  eigenfunctions  and  eigenvalues  and  it  is  shown  that  the  variances  of  the  superresolved  date 

Zel  DUC  t0  the  ,ack  °f  closed-form  expressions  forT  dgZectorf 

5n!S."  n0‘ pos!ible-  ™s  crb  =pp"»ci 

The  forward  model  describing  the  functional  dependence  of  the  image  i(x)  on  the  noise 
free  object  o(x),  the  system  PSF  h(x),  and  the  additive  noise  n(x),  is  given  by 


X„ 

i(x)=  Jh(x-oc)o(a)da  +  n(x), 


(1) 

where  hCx)  is  the  inverse  Fourier  transform  of  an  ideal  low-pass  filter  with  a  cutoff  frequency 

R5W8T11  IHs  sSrfn8t  n(X)  i®.stationary  ^re-mean  Gaussian  noise  with  variance  a 
ShS  '  1  f^Shtforward  to  generalize  this  forward  model  to  include  other  PSFs  and  noise 
models,  but  this  simple  and  oft-used  image  model  is  chosen  for  purposes  of  clarity.  The 

eigenfunctions  m— 0,1,2,...  ,  of  this  forward  model  are  the  celebrated  prolate 

spheroidal  wave  functions  (PSWFs)  of  Slepian,  Pollack,  and  Landau  [9]  M01  and  are  a 
function  of  the  space-bandwidth  product  (SBP)  4x0f()  [8],  Using  these  PSWFs  and  their 
associated  eigenvalues  Xra ,  Eq.(l)  can  be  rewritten  as 


i(*)=  £*»?»(*)  Jofe)?m(^+n(x), 


(2) 


where  the  PSWFs  are  normalized  to  unit  energy  in  the  interval  [-00,00]  and  the  eigenvalues  are 

zeTfor  at  °mb  mU6'-  ^  °,bjeCt  ***  fmite  SUpport’ 1116  eigenvalues  arf  greater  than 

,  ^  b  t  keir  VaIues  are  a  decreasing  function  of  m  and  converge  to  zero  (in  fact 
the  eigenvalues  are  the  energies  of  their  associated  PSWFs  in  the  object’s  support)  For  this 
reason,  the  inverse  of  Eq.(2)  exists  but  is  discontinuous. 

Since  our  interest  is  in  the  properties  of  estimates  of  the  object’s  Fourier  spectrum  outside 
£ ^  “fa“nnrndrdth’  WC  f0rmUlate  the  “  Problem  in  terms  of  estiiSg  £ ££ 
exprZonTg“en(by  SpeCtrum  I(f)  =  +  ^  desired* 


(3) 


/  \  m  f 

°e  f  =  +  jNfe)(pm(0^ , 

-^o  m=0  _f 

eigenvalues  are  the  enemies  nf  the  pQrrnjc  •  j  t  *wrs  ln  ^-(A)  an<i  their  associated 
m  (f\  u  ■  r  v  th£  PSWFs  lnside  the  measurement  bandwidth  [91  Because 

,  '  haS  mfimte  support  for  each  va*ue  of  m,  the  summations  in  Eq.(3)  must  be  over  all 
values  of  m  m  order  to  obtain  an  unbiased  estimate  of  n  rn  at  «- 
practical  application  can  include  nnlv  a  «„•*  ,  e  ^  at  any  frequency.  Although  any 

Eq.(3),  yielding  ^  “d  C3D  be  obtamed  in  a  straightforward  manner  from 


var{Ot (f )}  A  e[oc (f )0;  (f )]- 1 E[oc  (f ft (ff 

=  °2I>X(f) 


(4) 


m=0 


0oraSSOTF„™l'ra“"  SwtSf  SBPh  l”ve  T1"  proper,i“  wtal  w»»iy 

taide  the  °? iK  «“»- 

msSBP  is  outside  the  measurement  baudjdth!  We  fatl  ev^  [ewr  "J°5e  ”dex 
m^SBP  a  superresolvinz  PSWF  for  thic  rPoCrtri  x  .  .  .*7  whose  index  satisfies 
PSWFs  it  ic  ucsafiii  t  i  S  reason-  Po  8am  insight  mto  additional  properties  of 

2EH5 

Lf  of4„.  ^ 

measurement  bandSdtir.  Their  ^odatTelglnSes  *6 

becomeP™S?uf  prev.ious  para!,-lal,lL  “dee  that  the  supeiresolviug  PSWFs 

SLsrsjjy.'se  „fo“  s??e  >r“ f- h  *•«  *2 

meZSuTbaidr  SVc l  em  ^gy'Sfc 

SS52S2”  the  ,~OT«  '’“dwidd,  pennits  sup=™olvtog°  this'kZwWglm 
variedTs  tetov  Pr0,r“'S  ,lPSWFs  “d  ei8“™lues  as  the  SBP  is 


1 ^  SNRS  °f  °e(f)  f0r  aU  fr^ncies,  recall  that  the 
firL  d  °f  °E(f)  ls  Just  the  object  spectrum.  Therefore,  for  the  SNRs  to  remain 

or  not  var  jO  mi  T  finheT  ^  ^  *  Can  be  S6en  11131  the  question  of  whether 

Tat-  i  /fLe  °  „  f  f  any  frequcncy  1S  determined  by  the  index  dependence  of  the 

•  From  F'g-  l>  ‘t can  be  seen  that  <(f)->  0  for  any  frequency  fas  m  ->  oo, 

and  from  Fig.  2  that  ->  0  as  m  ->  oo.  As  a  result,  their  relative  rates  of  convergence’ 

detemnne  whether  or  not  var{Oe(f)}  remains  finite.  Because  there  does  not  exist  an  analytical 

expression  for  cpra(f),  Eq.(4)  cannot  be  used  to  definitively  answer  this  question.  However 

TlueTon  ^  PSWS  Tn-3oigenValUeS  caIculated  for  *&■  I  and  2  to  calculate  var{Oe(f)}  for 

ra£ 

ontl-d!  n  lnside  the  measurement  bandwidth  is  significantly  less  than 

outside  Because  of  this  fact,  it  appears  that  the  estimator  given  by  Eq  (3)  can  onlv  nrov  de  an 
oatoed  of  aoy  ftoqu^y  of  a  sopport-co^'L 

The  conclusion  that  an  unbiased  estimate  of  a  support-constrained  object’s  Fourier 
spectmm  at  any  frequency  has  infinite  variance  was  based  upon  a  spedfic  St^  fi  e 
Eq.(3)).  It  is  natural  to  wonder  if  an  alternate  estimator  exists  that  can  produce  an  unbiased 
estimate  with  fimte  variance  for  at  least  some  frequencies.  To  generate  an ^algoriZ, 

“timatorTunbiaTdeS  “f  T*™  ^  8  ^  appr°ach  t0  calculate  the  variances  of  any 
estimator  s  unbiased  estimate  of  a  support-constrained  object’s  Fourier  spectrum.  Y 


Frg.  2.  A  plot  of  the  PSWF  eigenvalues  associated  with  the  PSWFs  in  Fig.  1. 


Fig.  3.  A  movie  of  the  Fourier  variances  defined  bv  Eo  C4i  fnr  a  <;np  nf  an 

£35  of  frequency  ,o  «  ,he  Sfi CeSST  » 

on  the  l!r  TTr"*  ‘°  lncreasln8  the  fin',e  summation  limit  in  Eq.(4)  by  one  The  plots 
on  the  left  are  for  all  frequencies  included  in  the  calculations,  while  theplots  on  the  rich!  a e 1 
magnified  version  of  the  plots  on  the  left  for  frequencies  within  the  measurement  bandwidth. 


3.  Cramer-Rao  Bounds 

rlr  an  indePendent  approach  for  calculating  a  particular  set  of  lower 

unlrrd  estimate  °f  o(f)  is  emPio'ed  ^ lo-r  boi: 

fFTMt  LRB  3i-  diagonal  elements  of  the  inverse  of  the  Fisher  information  matrix 

corresponding  to  0{J  and  the  conditional  measurement  probability  density  function 

%  ^rmetenZed  by  0(f)'  °ur  aPProach  t0  calculating  these  CRBs  is  to  S 

fhe  Tnvert6  oflh^S  ^  ^  ^  take  illVerSe’  transform  this  inverse  into 

Althonoh  thJ rnn  corresponding  to  0(f),  and  then  extract  its  diagonal  elements 

though  the  CRBs  could  have  been  calculated  by  creating  the  FIM  for  Off)  directlv  it  will 
be  shown  that  the  transformation  process  from  the  image-domain  inverse  FIM  to  the  Fourier 
donuun  mverse  FIM  provides  important  insight  intone  properties  of  SVSSi  dZ£ 

varm?rlU5Ted  ^  ^orresPonding  to  a  vector  of  parameters  are  lower  bounds  to  the 
ances  of  any  unbiased  estimates  of  these  parameters.  CRBs  for  a  function  of  these 


fs  known.  ^ t0  ±e  V*auMm 

and  in  other  cases  they  can  be  aDDroachpd  ^in  j  Te’  m  many  cases  they  can  be  achieved 
parameters,  the  PDF  of  the  uncertainty  in  thp°m  ^  generate  tbe  CRBs  for  a  vector  of 
values,  must  be  known.  For  the  Wins  mnHpfTwn  condltjoned  on  the  parameter 

obtained  from  the  noise  PDF  and  the  noise  free  imaof*1'  1  ’/f?8  C°ndltional  PDF  can  be 
theory  applies  to  a  vector  of  ™  ^  ^gmS  model  However.  because  CRB 

rewritten  in  a  sampled  data  format  For  this  reason^  2  StT°Cha.fic  process’  E<h0)  must  be 
set  of  evenly-spaced  locatio^^^  *  by  a 

locations  rj>t  n  ,  .  .  .  tervai  [-x0,x0j  tor  some  arbitrary  number  of 

otuT^ZTJ  ?e“  ,oca,iT'  “d  let  y- 6-  “d  ”  •»  —  **>•< 

matrix  associated  with  h(a)  rS  Thk  2’  ^  1x1  addition>  iet  »  be  the  circulant 

given  by  h  permits  rewriting  Eq.(l)  as  a  matrix-vector  equation 


y  =  H0  +  r\ , 

where  the  PDF  ofy  conditioned  on  9  is  given  by 

f(y;e)=f.,(y-He), 


(5) 


(6) 


and  fj(r]-H0)  is  the  joint  pdf  of  r\  but  with  mean  H0 

unbiased^stimt^of  ol^is^to'calcidat^the^IM^1  **  "d  ^ 1  *  “  * 

m  in  the  p‘  row  and  LI  J  “oiit “itr  P  §  “  <*•  F<9>  ^  *“■<  ■* 


n,(e)=E 


dlnf(y;9)  ginf(y;8) 


59q 


(7) 


ssi  v,lue  of  ““  — *  ^ 


fp  (0) = — y  Ja(H6)k  d(He)„  ] 

>1  50p  50,  } 

=  ^rZh(“i  “ap)h(ak  -ap) 


(8) 


£?  fSSSS  $££  t ?  r  ?  ten““f  »  “d  *•  vector  of  x  lotions  a 

c„co,r  F(6).  Nohoe4 :  r“;sr"g  ofhowwe 

T  pIf  "™s?Cre,f  appro™d“”  «>  the  integral  of  He  pro™ of  rt^fS  ve, ™5 

ssst 

integration  implied  “  is  “  “T  «  the  toll 

intention  routine  based  upon  the  Numerical 

each  caseTtatS  1“ °f  Ifi  Prior  to  this  inverse  for 

A  set  of  eigenvalues  for  T  °  “f !l™“  “»  slabilitV  °f  hri-eme. 

vector  size  of  in'? a  1  *  j  f  parameters  as  the  results  in  Section  2  (SBP  of  40 

as  a  function  of  ti FM dge“ ate  Mex  T°"  f'°f  16‘  eI'“»K)  ™  plotted  in  Fig.  i 
me  rim  eigenvalue  mdex.  Several  important  issues  are  apparent  from  this 


similar  to  the  PSWF  eigenvalue  behavior  as  ^  /u  h  °f  ^  b,IM  ei8cnvalue  index  is  quite 
eigenvalues  are  close  to  one  for  values  of  the  FfU  °n  °f  ^  jSWF  mdex  m  Aat  the 
approximately  the  same  rate  as  do  the  kwF  ™  fIgenvalue  mdex  *  SBP  and  decay  at 

of  the  behaviors  and  value of Z  55£' Second,  because  of  the  similarity 
reasonable  to  assume  that  the  FM  eiSue7cZ  ^  **  ™  eigenva1^-  *  i 

inverse  of  F(0)  cannot  be  calculated  prtrtl  ^  ge  t0  Zer0‘  For  this  reason>  a  true 

pseudoinverse  of  F(0)  [16].  However,  ^  **  Calculated  usin8  a 

that  unbiased  estimates  with  finite  variances  f  n  n,  pseudomverse  must  be  used  implies 
As  a  result,  only  functions  !f  0  h okT'T  “  ®  “  “*  P°SsMe  ™ 
with  finite  variances.  Fortunatelv  m.H  f  b  estunated  m  an  unbiased  manner 

interested  in  the  Fourier  transform  of  0.  l£,Ta£Z  dZT™  ^  *  Sp“i<!c*11)''  w'  ■* 


“sr?,  -  « as  a”  s  £ 

identified  by  the  fact  that  itsSCRB  ri  P°Sfb.le  Wlth  fuute  variance,  then  it  can  be 

F(0)+  converges  to  a  finite  °f  ^  number  °f  ’ eigenvalues  included  in 

increases  UC  38  *e  number  of  F1M  eigenvalues  included  in  F(0)f 

8'  '  101  toe  same  set  of  parameters  as  used  for  generating  Fig.  4. 


m'  w'TwI !:  “S  7  Rvalue  •“  <*  »'culali„„  of 

file  first  is  that  none  of  the  CRBs  apneajT>° b7n°  icrva,10f|s  c‘i11  ,t)e  nlil<ic  horn  this  movie, 
inside  the  measurement  bandwidth  donot  annear  to  h?  p”*8  t0  *  fimt®  va,ue-  Even  the  CRBs 
CRBs  inside  the  measurement  bandwidth  i^unrin  kt  '?on''er8in8-  slow  increase  in  the 
CRB  values  outside  the  dUB  *  **  “leakaSe”  of  ^  ^ge 

about  by  the  Fourier-domain  correlations  InforaHv^LT^T^  bandwidth  brouSht 
outside  the  measurement  bandwidth  ar  •  •  by  th  supPort  cons  tram  t.  The  CRBs 

eigenvalues  included  in  the  calculation  of  KsT  ne's«o  d  “h  "  °f  the  FftI 

plots  in  Fig.  5  are  remarkably  similar  to  the  noieva^ce^o”  0ftgT°°  ""  ^ 


Sotted  ItTtoSS  off5  f0r  any  Unbiased  CStimatc  of  any  Fourier  spectra  for  a  SBP  of  40 

EEm.  5SZ&  firs, sK  to„r at  « the  —22 

pseudoinverse  of  the  FIM  The  plots  on  the  left  m8  °ne  additional  eigenvalue  to  the 
calculations,  while  the  plots  on  thJririit ?Je/or  * 1  Fluencies  included  in  the 
frequencies  within  ^  P’0tS  ««»  » 

Fourier  spectra  under  consideration  because  the  measuremmtnoi^is'sfgn^'iTidqjendent.80*118' 


that  rS TsSetf  2rlySiS  35  WeU  as  the  reSuItS  “  Section  2,  it  seems  clear 

variant  Sse  vTes  of M&JZ  **emSOh°1  ?T“Cy  iS  possibl' 
measurement  dimctlv  the,  c,„  n.  ^  the  measurement  bandwidth  are  available  from  lie 

merely  b“  1^^  £  esfint le  Sf  “  “  “  ^7  “™"  variances 

uubiaidsupemLwonwirS  v.ie??SS  "T"'’ 

are  explored  in  the  next  section  lend.n  t  P  Slble-  For  this  reason,  biased  estimators 
superresolution.  ’  kadmg  t0  4116  ConcePts  of  P^iary  and  secondary 

4.  Primary  superresolution 

.*t,blfes^ltrb,td^  ““  vgrfauces  is 

biases  are  a  auktttatTftoSX^^S  7*7  “  achi<,vable'  Be““sc 

FFla^’  some  properties  of  prhnaiy^ superres^utiou  arepresented  " 

finite-SSm  So SSpsOT  Z  d'™'0'  77  sped,iei  We  “  ““  * 

reasons  The  first  k Z  V  PSWF-based  inverse  problem  fonnulation  in  Eq.(3)  for  three 
to  the  inverse  problem  cau  be  see.  from  E,.,4?  SiShSZ  ZSe  SSS 


inversely  proportional  to  the  PSWF  eigenvalue  ,u 

of  the  PSWFs  inside  the  measurement  bandwSth  rh,  ^genva,ues  are  energies 

for  any  set  of  M  basis  functions  are  minimiLl suPerres°lved  data 
reason  is  that  the  similarities  between  the  mendr.  ^  ®  SWFsas  the  basis  set.  The  second 

variances  for  this  estimator  shown  in  Kg  3  i^plTSthePSvST'  ^  “  **  5  **  ^ 
sense  qualitatively  optimal.  The  third  reason  il  .u  *  ,  PSWF-based  estimator  is  in  some 
problem  permits  straightforward  analysis  of  the  W  closed-form  solution  to  the  inverse 
To  begin  the  analysis XfoTlfo (3)  ££ 1  TF***  °f  SUPOTesolu^. 
estimator  are  a  function  of  the  object  being  P™perties  of  this  (and  any  other) 

superresolved  data  produced  by  this  estimafor  ^°Wever>  lf  the  biases  in  the 

independent  of  the  object  being  estimated  the  re«,Tt  d  bf,i  cdiaracterized  m  a  way  that  is 
useful.  Such  a  characterization  is  possible  and  k  n  W°U  a  &  mfre  gcnend  and  dius  more 
behavior  of  the  PSWFs.  Because  the  estimator  i Ftfm*1  baSed  Up0n  the  frequency 
linear  combinations  of  PSWFs  it  follows  that  pqwu  generates  an  estimate  of  0(f)  using 
frequency  contribute  little  ?imal  ^  a*  a  given 

Fig.  1  shows  that  the  ^  m0Vie  °f  ^  PSWFs  “ 

region  outside  of  but  adjacent  to  the  measurement  h  a  •i.f  Cl°Sf  to  zer?  in  a  connected 
is  an  increasing  function  of  the  PSWF  indev  Th  andvt'ldtb,  and  that  the  size  of  this  region 

■he  bias  a  |ve.  superresojved^equency  R  ZSToftol  W°  fi<? 
contamed  in  PSWFs  not  included  k  *i!i  y  .  ctlon  of  the  energies  at  that  frequency 

function  of  the ‘Zt  E,  <3),“?  tha‘  ^  bi“  i!  • 

the  PSWFs  no,  included  to  S  STT’1™'’  *'’?“?■  if  <* 

be  essentially  unbiased.  gngible,  the  superresolved  data  can  be  said  to 

inverse  problem  mu's't  bfcdcukted  Knction  T*8*”  of,the/SWFs  not  included  in  the 
upper  limit.  To  present  the  results  with  Creased  ^  PSWF  index 

energies  of  the  PSWFs  at  each  superresolved  f™J  ^  these  sums  were  divided  by  the  total 
included.  The  results  of  these  calculations  .,r  d  ?equency  t0  show  the  fraction  of  energy  not 
of  the  movie  is  for  “  the”°vie  *  p%-  6,  where  each  frame 

PSWFs  used  to  g JS XEJS  US?  f°r  11118  movie  are 

similar  for  all  ‘sRPc  tv.  •  *  ,g‘  ’  bu  die  general  shapes  of  the  curves  in  Fie  6  are 

the  PS^  «  -t  idenfreal  for 
the  inverse  problem  for  a  given  sunemsoWf-d^^  ^  PSWF  energy  not  includcd  in 
PSWF  index  and  asymptotefto  zero  The  frfqUf Cy  decays  raPidly  as  a  function  of 

point  tha,  is  k  -r  the 

this  reason,  we  call  the  superresolved  dak  *  y  me  PSWF  energy  is  not  included.  For 
total  PSWF  energy  is  mcS  fo  the  f?  esfntially  unbiased  when  more  than  98%  of  the 

thus  have  quantified  the  phrase  ‘wLaii  i?"  °^hS  °bjCCt’S  sPectrum  at  that  point,  and 
arbitrary,  the  conclusions  to  be  drawn  fromV^d  r  ^though  tbe  98%  Point  is  somewhat 
relatively  insensitive  to  the  exato  jwiwntage^s^'in^i^efitotion!1111113^  are 
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Fig.  6.  A  movie  of  the  normalized  residual  energy  of  the  PSWFs  not  inrlmw  ■ 
reconstruction  process  as  a  function  of  the  PSWF  index  for  a  SBP  of  40  ,",fhe 

corresponds  to  a  value  of  0  02  Each  frame  3  0f  40'  The  dotted  hne 

superresolved  frequency  as  indicated  irnhe  movfe;  COrresPonds  10  »  different 


80 


Some  properties  of  primary  superresolution  will  now  be  diseinjcpH  p.Vct  *  *  a 

Became  to  maximurn  index  *  chosen  is  L  tchl  of  to X,  Ms  d„T“  S%Z 

"s  r“ of 

Ss'  ^dtoicniai'propert*1*6^1*011 

where  the  amount  of'nrim^68  °f  pnma7  Resolution  can  be  seen  by  examining  Fig.  7 
square  reotTZ  p^WP  ^  SUPerresolutlo»  *  Plotted  as  a  function  of  the  inverse  of  the 

olthe  estimated  FoS  ~re 

to™  ’it'rertcdT"  riffr  TrrcsoM°nr  p“,ibie  as  a  fiinc,io"  °f 

to“nSc?toi! to  XTco'V‘?7  0tll“  ”umber»fl J«8^™VeerSedBto°™e° 

J1*18  context  a  degree  of  freedom  refers  to  the  classical  sampling  degree  of 

R«co^ctiSaS  ““  °f  tW°  timeS  ^  ““  fflSK 

first  fsTaffmdSBP?iin°thert'eS  °f  ^perxesolutton  can  be  deduced  from  Fig.  7.  The 

the  SRP  to  rtfBP  !he  amount  of  pnmary  superresolution  is  essentially  independent  of 

frsrsrst is  °dT 1  ~ 

bandwidth  as  lorn*  L  th*  L  ■  ?  ,  e  ,  the  Founer-domain  measurement 

superresolution  possible  is  independent  of  thTsBP  fheT^ 

previous  result  £1];  however,  to  apparen!  conhadicton  i 


JESS  !!rTUred'  We  measure  ^resolution  in  terms  of  the  number  of  degrees 

IE e  ™,oach  X  »  ““  * besl  ™imKJ  « 

t™1 £-  rvc  dT“"“ 

superresolution,  however,  the  defimtion  of  degree  of  freedom  changes.P  Y 


c 

o 


7L,  ^  pl?  of1thetincreases  in  the  sampling  degrees  of  freedom  in  a  superresolved 
SfoTa  SBPofl  Ae°dUt  wt011"11?  Primary  suPerreso!uti°"  components.  The  dot-dash 
!he  dotted  £  for  a  “o  "  'S  ^  a  SBP  °fl0  ’ the  S£»id  «"*  »  **  *  SBP  of  20,  and 


5.  Secondary  superresolution 

The  concept  of  secondary  snpcrresolution  is  discussed  in  this  section  and  its  properties  are 
compared  and  contrasted  with  the  properties  of  primary  superresointion.  TteZoZZ 
superresolution  possible  when  including  both  primary  and  secondary  superresolved  data  is 
evaluated  using  a  PSWF-based  degrees-of-freedom  approach.  ItTZT St  second 
superresolution  is  a  more-than-additive  phenomena  and  that  the  amount  of  superresolution  is  a 
function  of  location  in  the  object  support,  unlike  primary  superresolution 

essentiX  0lS°n  “  38  aU  °f  *»  ^resolved  data  that  is  not 

essennahy  unbiased.  Another  way  to  express  this  fact  is  to  call  secondary  supeiresolution 

ei  her  biased  or  incomplete.  For  this  reason,  the  classical  degrees-of-freedom  relationship 
relatmg  the  number  of  independent  components  in  the  image  domain  with  the  maximum 
frequency  included  in  the  reconstruction  process  no  longer  holds.  To  illustrate  this  fact 

Xt1 1  Superrffed  F,°“ner  Spectra  of  a  simulated  !-D  triple  star  shown  in  Fig.  8.  The 
triple  star  was  created  in  a  1024  element  vector  and  consists  of  three  equal-magnitude  ooint 
sources  located  at  elements  437,  512,  and  552.  The  true  Fourier  ampXdes SSlE 
with  the  reconstructed  Fourier  amplitudes  using  Eq.(3)  with  an  upper  limit  to  the  summation 


of  46  The  PSWFsfrom  Fig.  1  were  used  for  this  reconstruction.  Because  these  PSWFs  were 
created  using  a  SBP  of  40  the  upper  summation  limit  of  46  means  that  six  superresolving 
PSWFs  were  included  m  the  reconstruction  process.  Notice  that  the  region  of  primary 
superresolution  is  indeed  essentially  unbiased,  while  the  region  of  secondary  superresdS 
strongly  biased  Although  the  normalized  frequency  axis  in  Fig.  8  extends  only  to  twice  the 
measurement  bandwidth,  the  superresolved  Fourier  spectra  has  infinite  support  because  the 
structure  of  superresolving  PSWFs  is  that  of  decaying  sinusoids.  Clearly  infinite  amounts  of 
uperresolution  are  not  possible  based  upon  the  variance  analyses  of  Sections  2  and  3  Thus 
the  classical  degree-of-freedom  model  is  not  valid  to  characterize  the  resolution  properties  of 
secondary  superresolution.  An  alternate  model  must  be  used 


® ' .  plo.ts  of  *e  Fourier  amplitudes  of  a  triple  star  for  the  parameters  listed  in  the  text.  The 
solid  line  is  a  plot  of  the  true  Fourier  amplitudes  while  the  dotted  line  corresponds  to  a 
of  the  true  Fourier  amplitudes  using  Eq.(3)  with  an  upper  limit  of  46  and  a  SBP 
j  Joe  frequency  axis  is  normalized  to  one  at  the  edge  of  the  measurement  bandwidth,  and 
the  dashed  line  indicates  the  upper  boundary  of  the  region  of  primary  superresolution. 


It  has  been  shown  previously  that  a  PSWF  of  index  m  has  m  zeros  in  the  support  region 
used  to  generate  the  PSWFs  [9].  Based  upon  this  fact,  it  has  been  proposed  that  the  resolution 
provided  by  the  m  PSWF  can  be  characterized  by  the  average  distance  between  these  zeros 
[6J.  Such  a  definition  of  resolution  agrees  with  the  classical  Rayleigh  resolution  limit  when  m 
is  equal  to  the  SBP.  Based  upon  this  definition  of  resolution,  the  number  of  degrees  of 
freedom  in  a  reconstructed  image  using  PSWFs  is  equal  to  the  number  of  PSWFs  included  in 
the  reconstruction.  This  definition  is  consistent  with  the  general  concept  of  resolution  in  the 
classical  sense  and  does  not  predict  infinite  resolution  when  including  an  infinite  number  of 
frequencies  as  is  the  case  for  the  sampling  degree  of  freedom. 

Using  the  PSWF  definition  of  degrees  of  freedom,  the  increase  in  the  resolution  of  a 
reconstruction  brought  about  by  including  both  the  primary  and  secondary  superresolution 
components  in  the  reconstruction  process  is  plotted  in  Fig.  9  as  a  function  of  the  inverse  of  the 
square  root  of  the  PSWF  eigenvalue  and  for  several  values  of  the  SBP.  There  are  several 
properties  of  the  increased  degrees  of  freedom  brought  about  by  including  secondary 
superresolution  m  the  reconstruction  process  worth  noting.  The  first  is  that  poorly-resolved 
objects  still  benefit  the  least  from  superresolution  process  as  measured  by  increased  degrees  of 
freedom.  The  second  is  that  the  amount  of  increase  in  degrees  of  freedom  is  no  longer 
independent  of  the  SBP  for  SBP  fe  10.  In  fact,  the  number  of  degrees  of  freedom  in  a 


SKE  «-*-  of  .he  of  d.grees 

«*”«**«.  process  Z  SST  tenTo 7  ■»*»  to 

superresolution  given  in  Section  4  For  JJ  f  0ur  defm,tlon  of  additive 

additional  degree  of  freedom  when  date  SNrT lOOfoTSl S^pITeS?1Uti0n  Pr°dUCeS  °ne 
combmation  of  primarv  and  seronHa™  ,  .  1  a11  SBP  values  &  10,  while  the 

freedom  when  the  SBP  is  10,  and  fi7e  addftfanal  de°n  pr°d^S  four  additional  degrees  of 
Thus  i,  appears  to,  seeondiTLp^otr,  <*«  to  SBP  is  40. 

supeiresolved  recopstocta  as  compared  ,o  meWipgj^, 'ptia^s^ ,1,“°“"  fa 


c 


rig.  y.  Plots  of  the  increases  in  the  PSWF-defined  deoi™c 

support  region  brought  about  by  including °,f  freedo1m  averaged  over  the  entire 
reconstruction  process.  The  dot-dash  line  is  for  a  SBPofT dth^r>n^7i  ?upfnJ5Solution  ™  the 
the  solid  line  is  for  a  SBP  of  20,  and  the  ffitoKr a SB?  of  40  ^  'S  3  SBP  °f  ,0- 


spacSg  oSzlroTTc^s  Ihe^tiresf  ‘ Resolving  PSWF  in  terms  of  the  average 
throughout  the  support  region  As  can  be°seen  11115)1168  **“*  •' Zer0S  are  evenly  sPaced 
Fig.  10,  this  is  not  the  case^  The  zeros  are  relative^  !  representadve  saperresolvmg  PSWF  in 
region,  but  their  spacings  decrease  ranirim  y  evenly  spaced  m  the  center  of  the  support 

implies  that  superresolvfng  PSWFs  produce^XrTT?68  °f  ^  SupPort  ™s 

region  and  lower  resolutioL  away  £ £Vd£ ^  of  SUPP°« 
9.  To  take  the  space-dependent  resohmw,^  J ftheJuPP0^  region  than  predicted  by  Fig. 

the  support  region  was  divided  into  two  reg^  fe^me^Tandti!8  PSWFS  ^  8CC0Un1’ 
edges.  The  amount  of  total  superresolution  l  75  /o  d  the  remam™g  25%  at  the 

is  plotted  in  Fig.  1 1  and  cXllt.  r  ”  hc  center  75%  of the  support  region 
entire  support  rfgion  fa  oSer  to  «  H  ^  leSS  *“  predicted  ^  FS  9  for  the 
original  degrees  of  freedom  in  the  Image  'the  th|;seco“dary  superresolution  results  to  the 
increase  fa  the  number  of  degrees  of  freedom  ?w tS,? otted  m  F,S-  11  correspond  to  the 
the  resolution  calculated  using  the  center  75%  of  0,7°^  °?CUr  “  the  entire  suPPort  region  if 

entire  support  region,  fa  addition,  notice  that  th^moSrtof  I 3°" 1S  ^  "f™  throu«hout 
independent  of  the  SBP  for  SBP  >  10  just  as  is  f  suPerresolution  is  essentially 

is  additive  by  our  definition  As  before  the  am  e  primary  superresolution,  and  thus 

for  SBP  values  on  the  ord£  oto£ A dfeSS'L  7  SUperresolutio»  ^evable  is  less 
oer  or  one.  A  difference  from  the  primary  superresolution  results  is 


that  the  amount  of  superresolution  is 
in  the  reconstruction  process. 


a  little  larger  when  including  secondary  superresolution 


Normalized  Support 


Fig.  10.  Plot  of  the  PSWF  with  index  65  for  a  SBP=40  demonstrating  that  the  spacings  of 
zeros  for  superresolving  PSWFs  in  the  support  region  decrease  as  the  location  in  the  support 
gets  c  suPPort  boundaries.  Notice  that  the  zeros  are  relatively  evenly  spaced  in  the 

center  75%  of  the  object  support. 


c 


1/Sqrt(Eigenvalue) 


Fig;  1 1 '  of  tile  increases  in  the  PSWF-defined  degrees  of  freedom  averaged  over  the 

center  75 /»  of  the  support  region  brought  about  by  including  both  primary  and  secondary 
superresolution  in  the  reconstruction  process.  The  dot-dash  line  is  for  a  SBP  of  I ,  the  dashed 
line  is  for  a  SBP  of  10,  the  solid  line  is  for  a  SBP  of  20,  and  the  dotted  line  is  for  a  SBP  of  40 
The  vertical  axis  is  the  number  of  degrees  of  freedom  that  would  be  added  to  the  entire  support 
regton  if  the  resolution  increase  calculated  in  the  center  of  the  support  region  is  the  same 
throughout  the  support  region. 


The  amount  of  total  superresolution  achieved  at  the  edges  of  the  support  region  for  the 
remaining  25%  of  the  support  region  is  shown  in  Fig.  12.  Notice  the  large  increase  in  the 
numbers  of  degrees  of  freedom  at  the  edges  of  the  support.  As  for  the  results  in  Fig.  11,  the 
numbers  in  Fig.  12  correspond  to  the  increase  in  the  degrees  of  freedom  that  would  occur  in 
the  entire  support  region  if  the  resolution  at  the  edges  of  the  support  region  is  the  same 
throughout  the  support  region. 


c 


Fig.  1 2.  Plots  of  the  increases  in  the  PSWF-defmed  degrees  of  freedom  averaged  over  the  25% 
of  the  support  region  located  at  its  edges  brought  about  by  including  both  primary  and 
secondary  superresolution  in  the  reconstruction  process.  The  dot-dash  line  is  for  a  SBP  of  1 , 
the  dashed  line  is  for  a  SBP  of  10 ,  the  solid  line  is  for  a  SBP  of  20,  and  the  dotted  line  is  for  a 
SBP  of  40.  The  vertical  axis  is  the  number  of  degrees  of  freedom  that  would  be  added  to  the 
entire  support  region  if  the  resolution  increase  calculated  at  the  edges  of  the  support  region  is 
the  same  throughout  the  support  region. 


To  put  the  primary  and  secondary  superresolution  results  in  perspective,  it  is  useful  to 
consider  the  relative  percentage  increases  in  resolution  for  a  specific  case.  To  this  end 
consider  an  image  that  has  a  SBP  of  40.  If  the  data  SNR  allows  the  use  of  all  the  PSWFs 
whose  eigenvalues  are  greater  than  10‘4,  the  percent  increase  in  the  resolution  throughout  the 
entire  support  region  due  to  primary  superresolution  would  be  2.5%  because  primary 
superresolution  only  provides  an  additional  degree  of  freedom.  The  resolution  increase 
including  both  primary  and  secondary  superresolution  in  the  center  region  of  the  support 
would  be  5%,  a  factor  of  two  increase  over  what  primary  superresolution  provides,  but  still 
small.  Finally,  at  the  edges  of  the  support  region,  the  resolution  increase  including  both 
primary  and  secondary  superresolution  would  be  30%,  a  large  and  noticeable  increase.  To 
demonstrate  visually  the  impact  of  these  numbers,  three  reconstructions  of  the  triple  star 
whose  Fourier  magnitude  spectrum  is  plotted  in  Fig.  8  are  displayed  in  Fig.  13.  The  first 
reconstruction  uses  just  the  Fourier  data  inside  the  measurement  bandwidth,  the  second 
reconstruction  uses  both  the  measured  Fourier  data  and  the  primary  superresolution  data,  and 
the  third  reconstruction  uses  all  the  superresolved  data  along  with  the  measured  Fourier  data. 
The  two  rightmost  components  of  the  triple  star  are  contained  inside  the  center  75%  of  the 
support;  as  a  result,  the  amount  of  superresolution  possible  is  predicted  to  be  5%  for  all  three 
reconstructions  and  thus  essentially  negligible.  The  reconstructions  in  Fig.  13  confirm  this 


prediction.  However,  the  leftmost  component  of  the  triple  star  is  near  the  edge  of  the  support 
region  and  benefits  from  the  increased  resolution  brought  about  by  secondary  superresolution. 
As  predicted  by  the  plots  in  Fig.  7,  the  primary  superresolution  reconstruction  of  this 
component  differs  very  little  from  the  no-superresolution  reconstruction.  However,  as 
predicted  by  the  plots  m  Fig.  12,  the  reconstruction  that  uses  secondary  superresolution  data 
has  an  increased  resolution  of  ~30%.  The  increase  in  resolution  is  greater  on  the  side  of  the 
component  near  the  edge  of  the  support  as  compared  to  the  side  nearer  the  center  of  the 
support  because  of  the  decreasing  zero  spacings  of  PSWFs  as  one  moves  toward  the  support 
boundaries.  Although  the  increased  resolution  is  encouraging,  it  can  be  seen  that  the 
reconstruction  using  secondary  superresolution  data  produces  less  accurate  estimates  of  the 
relative  magnitudes  of  the  triple  star  components. 


Fig.  13.  Plots  reconstructions  of  the  triple  star  whose  Fourier  amplitudes  are  shown  in  Fig. 
8,  where  the  solid  line  is  a  reconstruction  using  just  the  Fourier  data  inside  the  measurement 
bandwidth,  the  dashed  line  is  a  reconstruction  using  both  the  measured  Fourier  data  and  the 
primary  superresolution  data,  and  the  dotted  line  is  a  reconstruction  using  the  measured  Fourier 
data  and  both  the  primary  and  secondary  superresolution  data. 


6.  Discussion  and  Future  Work 

The  results  in  the  preceding  sections  provide  insight  into  the  properties  of  superresolved  data. 
It  was  shown  that  these  properties  can  be  seen  more  clearly  if  the  superresolved  Fourier  data 
region  is  decomposed  into  two  separate  regions.  The  first  region  is  connected  to  and  outside 
of  the  measurement  bandwidth  and  is  the  region  where  essentially  unbiased,  or  primary, 
superresolved  data  resides.  The  second  region  is  outside  of  both  of  these  regions  and 
connected  to  the  primary  superresolution  region  and  is  the  region  where  biased,  or  secondary, 
superresolved  data  resides.  Although  the  secondary  superresolution  region  is  theoretically 
infinite  m  size,  the  data  SNR  limits  it  to  be  finite. 

Primary  superresolution  is  additive  in  nature  in  that  the  amount  of  primary 
superresolution  is  independent  of  the  SBP  as  long  as  the  SBP  fc  10  and  depends  only  on  the 
data  SNR.  For  SBPs  on  the  order  of  one,  the  amount  of  primary  superresolution  for  a  given 
data  SNR  is  less  than  for  larger  values  of  the  SBP.  The  increase  in  resolution  in  the  image 
domain  is  constant  across  the  support  region.  In  addition,  for  objects  with  SBPs  a  10,  the 
amount  of  primary  superresolution  achievable  with  realistic  data  SNRs  is  on  the  order ’of  a 
few  percent  or  less. 


Secondary  superresolution  produces  increases  in  resolution  in  the  support  region  that  are 
a  function  of  die  location  in  the  support  region.  Across  most  of  the  support  region  away  from 
its  edges  the  resolution  increase  due  to  secondary  superresolution  is  essentially  constant  and 
has  many  of  the  same  properties  as  primary  superresolution  (i.e.,  the  amount  of  resolution 
increase  is  independent  of  die  SBP  when  the  SBP  a  10,  and  for  smaller  SBPs  the  amount  of 
resolution  increase  is  less).  In  addition,  secondary  superresolution  in  the  interior  of  the 
support  region  adds  more  resolution  to  the  primary  resolution  results,  but  still  is  on  the  order 
of  a  few  percent  for  data  with  SBPs  a  10.  These  properties  are  due  to  the  portion  of  the 
superresolved  data  in  the  (small)  region  of  the  secondary-superresolution  Fourier  data 
adjacent  to  the  primary  superresolution  region.  However,  the  amount  of  resolution  increase  at 
the  edges  of  the  support  region  due  to  secondary  superresolution  is  much  greater  than  in  the 
interior  of  the  support  region.  For  example,  for  data  with  a  SBP  of  40,  the  resolution  increase 
averaged  over  the  edges  of  the  support  region  is  ~30%.  These  higher  resolutions  are 
associated  with  the  long  “tails”  of  the  secondary  superresolution  data  in  the  Fourier  domain. 

Based  upon  these  properties  of  primary  and  secondary  superresolution,  it  is  clear  that 
both  schools  of  thought  regarding  the  achievability  of  meaningful  superresolution  can  be 
correct  depending  upon  the  metric  used  to  quantify  the  phrase  “meaningful  superresolution”. 
If  the  metric  regarding  the  achievability  of  meaningful  superresolution  is  the  obtainment  of 
more  than  a  few  percent  resolution  increase  across  the  entire  support  region  for  data  with  SBP 
ft  10,  meaningful  superresolution  is  not  possible.  If  the  metric  is  obtaining  more  than  a  few 
percent  resolution  increase  at  the  edges  of  the  support  for  any  SBP,  meaningful 
superresolution  is  possible. 

This  latter  point  is  worth  emphasizing.  It  is  well  known  that  the  human  visual  system 
keys  off  of  edges  in  an  image.  Therefore,  if  the  object  under  consideration  is  an  object  with 
well-defined  edges  on  a  black  background  (such  as  is  common  in  astronomical  imaging), 
meaningful  superresolution  appears  to  be  quite  possible  and  worth  pursuing.  On  the  other 
hand,  if  the  edges  of  interest  are  interior  to  the  object  support,  little  edge  enhancement  is 
possible. 

The  results  presented  in  this  paper  have  been  independent  of  the  actual  object  being 
imaged  and  superresolved.  It  is  important  to  determine  how  effective  primary  and  secondary 
superresolution  are  in  improving  the  resolution  of  images  in  terms  of  looking  at  actual 
examples.  Our  initial  results  (Fig.  13)  indicate  that  using  secondary  superresolution  in  the 
reconstruction  process  can  increase  resolution  at  the  edges  of  the  support  region  but  at  the  cost 
of  decreased  radiometric  accuracy.  In  addition,  our  results  only  included  noise  in  the  sense  of 
limiting  the  number  of  PSWFs  that  could  be  included  in  the  reconstruction.  We  expect  that 
noisy  data  will  require  additional  constraints  in  the  reconstruction  process  such  as  Fourier- 
domain  regularization.  For  these  reasons,  we  are  continuing  our  research  into  the  image 
domain  effects  of  secondary  superresolution  and  will  report  on  the  results  a  future  publication. 
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